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Abstract 

Background: Over two-tlnirds of UK medical scliools are augmenting tlieir selection procedures for medical 
students by using the United Kingdom Clinical Aptitude Test (UKCAT), which employs tests of cognitive and 
non-cognitive personal qualities, but clear evidence of the tests' predictive validity is lacking. This study explores 
whether academic performance and professional behaviours that are important in a health professional context can 
be predicted by these measures, when taken before or very early in the medical course. 

Methods: This prospective cohort study follows the progress of the entire student cohort who entered Hull York 
Medical School in September 2007, having taken the UKCAT cognitive tests in 2006 and the non-cognitive tests a 
year later. This paper reports on the students' first and second academic years of study. The main outcome 
measures were regular, repeated tutor assessment of individual students' interpersonal skills and professional 
behaviour, and annual examination performance in the three domains of recall and application of knowledge, 
evaluation of data, and communication and practical clinical skills. The relationships between non-cognitive test scores, 
cognitive test scores, tutor assessments and examination results were explored using the Pearson product-moment 
correlations for each group of data; the data for students obtaining the top and bottom 20% of the summative 
examination results were compared using Analysis of Variance. 

Results: Personal qualities measured by non-cognitive tests showed a number of statistically significant relationships with 
ratings of behaviour made by tutors, with performance in each year's objective structured clinical examinations (OSCEs), 
and with themed written summative examination marks in each year. Cognitive ability scores were also significantly 
related to each year's examination results, but seldom to professional behaviours. The top 20% of examination achievers 
could be differentiated from the bottom 20% on both non-cognitive and cognitive measures. 

Conclusions: This study shows numerous significant relationships between both cognitive and non-cognitive test scores, 
academic examination scores and indicators of professional behaviours in medical students. This suggests that 
measurement of non-cognitive personal qualities in applicants to medical school could make a useful contribution to 
selection and admission decisions. Further research is required in larger representative groups, and with more refined 
predictor measures and behavioural assessment methods, to establish beyond doubt the incremental validity of such 
measures over conventional cognitive assessments. 
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Background 

In recent decades it has been generally recognised that, 
in addition to academic ability, doctors' professional 
competence depends on a range of desirable personal 
qualities (for example [1]). These have been recently 
summarised [2] as teamwork and professional skills, 
duty and responsibility, professionalism and values, com- 
munication and interpersonal skills, and trustworthiness 
and ethical behaviour. It is also now recognised that 
medical schools have a duty to society to select as stu- 
dents those individuals most likely to graduate as doc- 
tors with such attributes [3]. 

To achieve this, some universities have been investi- 
gating the use of tests of cognitive and non-cognitive 
personal qualities to augment their selection procedures 
for medical students. In the UK 26 of the 32 medical 
schools have been exploring the use of the United King- 
dom Clinical Aptitude Test (UKCAT) [4]. The use of 
such tests is controversial and clear evidence of their 
predictive validity continues to be lacking [5-7] because 
studies combining the administration of these tests with 
observational measures of student behaviour (or profes- 
sional behaviour after graduation) have not been carried 
out. In addition the literature lacks reports of whether 
the non-cognitive characteristics desired in a good doc- 
tor have any bearing on students' examination results, 
particularly in performance-based exams. Our aim was 
therefore to establish whether any of a range of tests of 
cognitive abilities and non-cognitive personal qualities 
[8-11] that have been developed for medical student se- 
lection can predict professionally appropriate behaviours 
in medical students, in addition to their normal examin- 
ation results. 

At Hull York Medical School (HYMS), students' 
marks from their annual summative theme-based 
examinations can be broadly attributed to one of three 
domains: recall and application of knowledge, evalu- 
ation of data, and communication and practical skills, 
the latter including ratings from objective structured 
clinical examinations (OSCEs). HYMS tutors undertake 
regular formative assessments of students' professional 
behaviours, understanding and performance from 
the first year of the course onwards, in order to get 
closer to the ideal of a comprehensive assessment. 
Using a set of rating scales, these assessments record 
behaviours acknowledged as important by medical 
educators. This includes behaviours reflecting con- 
scientiousness (such as punctuality and appropriate 
dress), interactions with tutors and fellow students, 
involvement in the group learning experience, and 
appropriate clinical behaviours. The tutor ratings pro- 
vide a novel, but appropriate, set of outcome vari- 
ables against which to examine the predictive validity 
of selection tests. 



The range of information available from the HYMS 
assessments thus allowed exploration of whether the 
UKCAT cognitive and pilot non-cognitive tests of per- 
sonal qualities can predict either examination perform- 
ance or the surrogate measures of professional 
behaviours as observed by the tutors. 

Methods 

This paper reports the first two academic years' progress 
of the 146 students who, in September 2007, com- 
menced year 1 of the five-year medical course at Hull 
York Medical School (HYMS). All were invited to par- 
ticipate in the study, which had ethics approval from 
HYMS' Medical Education Ethics Committee (study ref 
0701). Data were collected from the following measures. 
Figure 1 shows when each set of measurements was 
undertaken. 

1. The UKCAT test, which all students were required 
to have taken in 2006 before applying to medical 
school, provided four cognitive skills subtest scores: 
verbal reasoning (VR), numerical reasoning (NR), 
abstract reasoning (AR) and decision analysis (DA) 
[4]. HYMS did not preselect students on the basis of 
any minimum or specified range of UKCAT 
performance. UKCAT results were available for 131 
students; all but two of the remaining students had 



Academic status 


Date 




Tests 


Pre-application 


2006 


July 


UKCAT 






to 


cognitive 






Sept 


tests 



Academic year one 



2007 


Sept 




Oct 




Nov 




Dec 


2008 


Jan 




Feb 




IVIar 




Apr 




IVlay 




June 



Non-cognitive tests 



Tutor assessments 
Exams 



Academic year two 



2008 


Sept 




Oct 




Nov 




Dec 


2009 


Jan 




Feb 




IVlar 




Apr 




IVlay 




June 



Tutor assessments 



Tutor assessments 
Exams 



Figure 1 Schedule of testing from pre-application to end of 
academic year two. Timing of cognitive and non-cognitive 
qualities tests, tutor assessments and end of year examinations. 
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applied a year earlier, before the UKCAT was 
introduced. 

2. The non-cognitive personal qualities assessments 
were three paper-based tests delivered under 
examination conditions at the University of Hull and 
the University of York in October 2007. The tests, 
which have also been part of the UKCAT since 2007, 
were: 

a) The Interpersonal Traits Questionnaire (ITQ), 
which measures narcissism, aloofness, confidence 
(in dealing with people) and empathy and 
produces a summary score for INVOLVEMENT 
(versus detachment) in which confidence and 
empathy are positive, narcissism and aloofness 
negative [8,11]. 

b) The Interpersonal Values Questionnaire (IVQ), 
which measures the extent to which the 
respondent favours individual freedoms (Versus 
societal rules) as a basis for making moral 
decisions [9,11]. 

c) The Self- Appraisal Inventory (SAI) [11], which 
measures the domains of (mental) RESILIENCE 
(comprising scales measuring anxiety, moodiness, 
neuroticism and irrational thinking) and SELF- 
CONTROL (versus risk taking tendency) using 
the scales of restraint, conscientiousness, 
permissiveness and anti-social tendencies. SAI also 
contains a Lie scale. 

3. Tutor assessment (TA) data are collected routinely 
about all students, from each problem-based 
learning (PBL) tutor. Groups of 8 students meet 
with the same tutor twice a week for a 1.5 h 
problem-based learning (PBL) tutorial throughout 
year 1, and again (in different groups with different 
tutors) throughout year 2. These tutors, all 
clinicians, are also the personal mentor for their 
students, and consider each individual's assessment 
data in formative one-to-one interviews between 
tutor and student. These data comprised: 

a. Assessment of a set of specified behavioural 
items, adapted from the Peer assessment of 
professional behaviours form created by Gary 
Butler, University of WoUongong, and used with 
permission. The items were suggested by the 
requirements for medical school curricula [3], 
taking into consideration the ability of tutors to 
observe student behaviour. The assessments were 
undertaken by their PBL tutor for each student 
once in year 1 (in May 2008) and twice in year 2 
(January and May 2009). The assessment form 
was being developed over this period; it was 



piloted in May 2008, revised in January 2009 and 
finalised in May 2009, so there was slight 
variation in the items on each occasion, with the 
tutors assessing 17 behaviours in May 2008 and 
May 2009, but only 14 behaviours in January 
2009. See Additional file 1 for an example of each 
form. The final three items (18 to 20) on the May 
2009 form are excluded from the analysis because 
these items relate to a different context (clinical 
placements) and were assessed by other tutors 
whom the students met irregularly, 
b. In addition, as part of this research, the PBL 
tutors were asked to make an overall assessment 
of each student in May each year, by rating them 
as either 'problematic^ 'average' or 'particularly 
promising'. 

4. The marks from end of academic year summative 
examinations, held in June 2008 and June 2009, 
which are allocated to one of three HYMS themes: 
Theme A {Life sciences and Clinical sciences). Theme 
B {Clinical techniques and skills and Person-centred 
care) and Theme C {Evidence-based decision- 
making. Population health and medicine and 
Managing resources for quality and efficiency). 
Theme A tests largely knowledge recall, with some 
interpretation. Theme B tests interpersonal 
understanding and communication and practical 
skills by a written paper (30% of marks) and an 
OSCE (70%). The OSCE included equal numbers of 
five minute stations covering practical skills and 
communication skills; for a detailed list of the 
stations, see Additional file 2. Theme C tests not 
only knowledge, but also analytic and numerical 
evaluation skills. 

Tutor assessments and examination data were avail- 
able for the whole cohort of 146 students. 143 agreed 
to take part in the non-cognitive study but only 137 
completed all three non-cognitive tests, of whom 122 
had also completed the UKCAT cognitive tests (see 
Additional file 3). 

The data were entered into SPSS for analysis and 
screened for abnormal distributions. As this was an ex- 
ploratory rather than a hypothesis driven study, struc- 
tured methods such as regression analysis and 
adjustment of significance levels for repeated compari- 
sons were eschewed in favour of describing basic rela- 
tionships between variables, and the effects of 
collinearity were not taken into account. The boundary 
provided by statistical significance was taken as a guide 
to which relationships are reported as important. 

Pearson product-moment correlations were computed 
within and between the groups of measures given above, 
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Figure 2 Relationships explored In text and tables. 



and significant results (p < .05, 2-tailed test) tabulated. 
Although only four rating categories (Unsatisfactory, 
Borderline, Satisfactory and Excellent) were available to 
tutors, these scales were assumed to approximate inter- 
val measurements and to be suitable for parametric stat- 
istical tests. Correlations involving variables with some 
missing data were automatically adjusted on a case-by- 
case basis. To assess relationships with high and low 
medical school performance, scores on non-cognitive 
tests, tutor assessment (TA) items, overall tutor rating 
and OSCEs were compared for students obtaining the 
top and bottom 20% of the summative examination 
results, using one-way Analysis of Variance to produce 
an F value. 

Results 

This study reports students' progress up to the end of 
year 2, including performance in the year 1 and year 2 
summative examinations. Figure 2 summarises the rela- 
tionships explored. The significant results (p < .05, 2- 
tailed test) among the Pearson product-moment corre- 
lations for each group of data are shown in the following 
tables, along with significant F values obtained by com- 
paring students obtaining the top and bottom 20% of 
the summative examination results using Analysis of 
Variance. In the interests of clarity, variables with non- 
significant results (personality and tutor ratings) are 
omitted and revealed as blank cells in the tables. 

Structure of the tutor assessment scales 

The structure of the tutor assessment scales was ana- 
lysed for each session using reliability analysis. The in- 
ternal consistency of the scales was high in all cases, the 
Cronbach alpha coefficients being 0.91 for May 2008 (17 
items), 0.87 for January 2009 (14 items), and 0.90 for 
May 2009 (20 items), indicating that there was a strong 
tendency for the tutors to give similar ratings to individ- 
ual students for all items in the scales. 

Prediction of tutor assessments by non-cognitive tests 

The significant relationships found between non- 
cognitive characteristics and tutor assessments in year 1 



and year 2 are shown in Additional file 4: Table SI and 
Additional file 5: Table S2 respectively. 

From the SAI test, high neuroticism (p < .05), moodi- 
ness (p < .05), irrational thinking (p < .05 to p < .001), and 
low conscientiousness (p < .05) predicted lower ratings by 
tutors for some individuals' behaviours in year 1. These 
traits combine into the SAI factor RESILIENCE, which 
itself predicts some positive behaviours of individuals in 
the first year PBL groups (p < .05). In year 2, the SAI fac- 
tor SELF-CONTROL (incorporating + restraint,-permis- 
siveness, -antisocial behaviour) (p < .05 to p < .001) and 
conscientiousness (p < .05 to p < .01) predict some posi- 
tive behaviours of individuals, while the individual traits 
permissiveness and anti-social behaviour both correlate 
negatively (p < .05) with some year 2 behaviours. 

All of the individual measures of the ITQ {empathy, 
confidence, narcissism and aloofness) correlate with some 
tutor assessed behaviours of individuals in year 1 and in 
year 2 (p < .05 to p <0.001) and, in general, the correla- 
tions are in the direction that is expected intuitively, for 
example narcissism and aloofness are negatively corre- 
lated with positive behaviours. The combined ITQ factor 
INVOLVEMENT {empathy, confidence, low narcissism, 
low aloofness) correlates positively with the overall tutor 
assessment in year 1 (p <0.05) but not in year 2. In year 
2 only, social responsibility (IVQ test) predicts appropri- 
ate self-management (rather than group-orientated) 
behaviours (p < .05 to p < .01). 

Prediction of examination results by tutor assessments 

The tutor assessments focus on a range of desirable pro- 
fessional behaviours. Many, but not all, items predict 
aspects of examination performance in both year 1 and 
year 2, as shown in Additional file 6: Table S3 and 
Additional file 7: Table S4. For both years, those tutor 
assessment items that related to the students' participation 
in the functioning of the group (for example, contribution 
of work to the group, and contribution to the positive at- 
mosphere in the group) best predicted overall examination 
performance (including OSCE skills), and best predicted 
the differences between the best and worst performing 
students in their examinations (top 20% vs bottom 20% of 
marks range). Somewhat surprisingly, there are more sig- 
nificant relationships with Theme A scores (recall of sci- 
entific knowledge) than with Theme B (clinical techniques 
and skills, including OSCEs). 

Prediction of examination outcomes by non-cognitive 
subtest scores 

The non-cognitive traits that predicted examination out- 
comes are shown in Additional file 8: Table S5. Greater 
narcissism and aloofness (from ITQ), and irrational 
thinking (from SAI), predicted poorer overall examin- 
ation scores (p < .05), and poorer performance in Theme 
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A, Theme B overall and communication skills OSCEs. 
Conscientiousness (from SAI), confidence (from ITQ) and 
the INVOLVEMENT factor score all predicted better per- 
formance in the practical skills OSCEs (p <0.05 to p 
<0.001). Narcissism (p <0.05), aloofness (p < .001) and the 
INVOLVEMENT factor score (p < .05) each differentiated 
between the best and the worst performing students in the 
year 1 examinations. Non-cognitive test scores did not pre- 
dict examination performance in Theme C. 

Prediction of tutor assessments by UKCAT cognitive test 
scores 

UKCAT cognitive tests scores did not predict many of 
the behaviours subsequently rated by tutors, as shown in 
Additional file 9: Table S6. In year 1, no behavioural 
items in the year 1 tutor assessments (May 2008) corre- 
lated with the overall UKCAT cognitive ability test score, 
and only one (negative) correlation was found with a 
single subtest score. Although the year 2 tutors under- 
took assessments twice, only eight significant correla- 
tions between the behavioural measures and cognitive 
test scores were found. The abstract reasoning test score 
predicted four behaviours in the May 2009 assessment 
by tutors. The UKCAT cognitive test overall score corre- 
lated with only one rated behaviour from the May 2009 
tutor assessment and no behaviour from the May 2008 
and January 2009 tutor assessments. 

Prediction of examination outcomes by UKCAT cognitive 
test scores 

The UKCAT cognitive test overall score and the decision 
analysis subtest score were significant predictors for 
both year 1 and year 2 overall exam scores (p < .001), 
and performance in Theme A (which tests knowledge 
acquisition) (p < .05 to p < .01), as shown in Additional 
file 10: Table S7. Both were also strong predictors in 
each year of the best versus worst examination perfor- 
mers. Performance in Theme C (which requires acquired 
knowledge and the application of numerical and analyt- 
ical skills) was the result best predicted each year by 
both the overall cognitive ability score (p < .001) and by 
each subtest score. Cognitive test scores did not predict 
performance in the communication skills components of 
the clinical examinations (OSCEs) in either year; phys- 
ical examination skills correlated positively only with de- 
cision analysis and only in year 1. Overall performance 
in theme B (patient centred-care, communication and 
interpersonal understanding) was predicted, but only in 
year 2, by three of the four cognitive test scores. 

Reliability of tutor assessments 

Additional file 11: Table S8 shows the re-test reliabilities 
of those behaviours assessed at least twice, by different 
tutors (year 1 tutor: May 2008; year 2 tutor: January 



2009, May 2009). The reliabilities of the overall tutor rat- 
ings are high (p < .001), as are the reliabilities of the items 
that relate to group characteristics, such as integrates into 
group, demonstrates appropriate attitudes, takes responsi- 
bility for group learning contributes work for the group, 
and willing to learn from others (p < .01 to p < .001). 

Discussion 

This study rests on the co-incidence of three fortunate 
sets of circumstances. First, it was possible to follow an 
entire medical school entry cohort of students, the ma- 
jority of whom had taken the UKCAT cognitive tests but 
the scores had not been used in the students' selection 
(so there was no restriction of range within the study 
population) and then for research purposes these stu- 
dents took the same non-cognitive tests that were 
included within the subsequent years' UKCAT. Secondly, 
the tutor assessments provide perhaps the closest ap- 
proximation to all-round evaluation of individual profes- 
sional performance during the early years of a medical 
course. Thirdly, the HYMS theme-based examination 
system requires allocation of examination marks into the 
three separate themes that depend mainly on scientific 
knowledge (Theme A), on clinical information gathering 
and inter-personal skills (Theme B) and on application 
of statistical and analytical skills to acquired knowledge 
(Theme C), thus allowing different aspects of students' 
performance to be distinguished. 

Principal findings 

This study has found numerous significant relationships 
between students' prior cognitive and non-cognitive test 
measures (all of which became components of the 
UKCAT), subsequent tutor assessments of individual 
and group-related behaviours, and academic and clinical 
examination results undertaken over the first two years 
of a medical school course. 

The individual predictive ability of any of the measures 
(of cognitive skills, non-cognitive traits and of behaviour) 
appears weak, the majority of the statistically significant 
correlations being in the range 0.16 to 0.24 and only a few 
exceeded 0.30 (accounting for only 9% of the variance). 
However, in the field of organisational psychology it is well 
recognised that even quite weak correlations are useful 
predictors of workplace outcomes, especially if the ratio of 
applicants to selectees is large [12], as with applications to 
medical school. Such measures may therefore usefully 
predict medical student performance and later profes- 
sional conduct. This position is supported by a recent 
meta-analysis of the predictive value of ability and per- 
sonality test scores, which concluded that such scores 
are more successful in predicting educational, work and 
life outcomes than is often admitted by critics; in par- 
ticular "In medical education, personality characteristics 
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gain importance for later academic performance when 
applied practice (such as performance in practicums 
and clerkships) increasingly plays a part" [13]. 

A further possible criticism of the findings is that there 
are relatively few significant relationships, as revealed by 
the number of blank cells in Additional file 4: Table SI and 
Additional file 5: Table S2 particularly. However, of the 
380 possible relationships that could have been included in 
these tables (19 personality scales by 20 tutor ratings), 51 
(or 13.4%) were significant at the 5% level or better; this 
becomes 18.2% of 280, if only the 14 personality scales ac- 
tually appearing in the tables are considered. When com- 
pared with the chance proportion of 5% it is clear that the 
overall pattern of results is better than chance. Additional 
analyses were conducted to adduce further evidence for 
the significance of the overall pattern, including multiple 
regression to calculate the proportion of common variance 
shared by the personality scales and the tutor ratings, and 
application of a recently published resampling technique 
[14] for calculating the probability of relationships between 
personality and behaviour. However, neither provided 
stronger support than the simple number of correlations, 
so the details are not presented here. 

In summary, although the statistically significant coef- 
ficients in the matrices could all be chance findings this 
seems unlikely because all of the significant results (and 
most of the non-significant ones, which are not shown) 
are in the direction that the particular personality traits 
and cognitive skills would be expected to affect the be- 
havioural and examination outcomes. 

The findings in more detail 

Overall, it appears that the non-cognitive tests do pre- 
dict normally unrecorded aspects of medical students' 
performance. Greater narcissism, aloofness and irrational 
thinking predicted lower tutor ratings for group-related 
behaviours, and poorer overall examination performance 
including in the most knowledge-based Theme A, as 
well as in Theme B (person-centred care), though not in 
Theme C. Good teamwork skills therefore appear to be 
an important student attribute. The finding that RESILI- 
ENCE predicted good group functioning in year 1, while 
conscientiousness and SELF-CONTROL predict this in 
year 2, may reflect students' increasing ease and familiar- 
ity with medical school. The early years of the HYMS 
course involves not only problem-based learning ses- 
sions but also a structured weekly programme of lec- 
tures, laboratory-based practical classes and tutored 
clinical experience, so the importance of group function- 
ing to the examination outcomes cannot be explained by 
exclusive reliance on problem-based group learning. 

While the UKCAT cognitive tests scores did not pre- 
dict many of the behaviours subsequently rated by 
tutors, they did predict both year 1 and year 2 overall 



examination scores, and performance in Theme A and 
Theme C. Notably, but not unexpectedly, cognitive test 
scores did not predict performance in the communica- 
tion skills or physical examination components of the 
clinical examinations (OSCEs) in either year, although 
they did predict overall Theme B scores at the end of 
year 2. This may reflect increasing emphasis in year 2 on 
clinical reasoning skills within this theme. 

Problems with predicting traditional medical school 
results 

Prediction of any outcome depends on selection and ac- 
curate measurement of both predictor (be that a cogni- 
tive skill or a personality trait) and an appropriate 
outcome (for example, academic marks, skills, or profes- 
sional behaviours). In order to examine whether any of 
the cognitive and pilot non-cognitive components of the 
UKCAT are valid predictors it is necessary to have ap- 
propriate measures of the behaviours that the tests are 
expected to predict. Paradoxically, in-course examina- 
tions do not usually assess outcomes that are related to 
the qualities that non-cognitive selection tests are 
designed to measure. In general, medical schools have con- 
centrated on traditional examinations (in part so that li- 
censing requirements can be fulfilled unambiguously) that 
test recall of factual material and reasoning, which depends 
on memory and cognitive skills. Non-cognitive skills, such 
as communication and doctor-patient relationships, are 
often tested informally within medical training, and the 
results recorded in pass/fail format, usually with only a 
small number of failures (as with many academic medical 
examinations). Thus a problem for statistical comparisons 
of traditional medical school results is that pass/fail clinical 
outcomes are categorical and typically severely skewed 
while the predictor test results are normally distributed on 
a continuous scale. The present study benefits from finer 
gradation in both selection tests and the outcomes mea- 
sured, and from better matching between predictor and 
criterion variables. 

Other studies 

Cognitive tests would be expected to predict success in 
examinations of knowledge recall. This has been shown 
in one recent study [15] about medical students, but not 
in others [5-7]. No reported studies have been found that 
tested the hypothesis that non-cognitive characteristics 
desired in a good doctor have any bearing on students' 
examination performance, although a Conscientiousness 
Index [16] has been proposed as a measure of medical 
students' professionalism based on a variety of routinely 
made behavioural observations that can be recorded in a 
systematic and reliable way. The present study is the first 
to demonstrate that measurement of a range of different 
personal qualities can predict different aspects of medical 
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Students' performance. The broad range of suitable mea- 
sures of non-cognitive skills and professional behaviours 
used here provides possible tools for future research. 

Relevance of this study: implications for clinicians and 
policymakers 

All health professionals require good communication 
skills and the ability to work effectively as part of a team. 
Doctors have additional roles, not only in education and 
in research but also, principally, as decision-makers and 
leaders of teams working in situations of clinical com- 
plexity and uncertainty. All doctors therefore should be 
committed to reflective practice, monitoring their own 
contribution and working continually to improve their 
own and their team's performance [17]. Selection tests 
need to encompass more than purely cognitive skills, 
and test for the other abilities that underpin the whole 
range of the doctor's professional tasks. 

In the present study, non-cognitive tests were found to 
predict behaviours likely to be important when working 
as a doctor, such as functioning well with others in 
groups, acknowledging weaknesses and accepting feed- 
back (which underpins the ability to learn and change 
through experience), and identified other traits, such as 
narcissism, aloofness and irrational thinking, that are 
likely to diminish a doctor's ability to fulfil these roles. 
Improvement in the predictive power of such non- 
cognitive tests depends not only on finding better tests, 
but on devising and using better measures of critical 
behaviours in medical school and professional practice. 

Unanswered questions and future research 

The findings raise three further questions with implica- 
tions for medical education. First, in the selection of fu- 
ture medical students, should the predictive ability of 
cognitive and non-cognitive qualities be explored more 
fully, particularly in relation to the difficult judgments 
and high level of inter-personal skills required of a doc- 
tor? Secondly, should medical schools be making use of 
more standardised and repeated behavioural observa- 
tions undertaken by tutors throughout the students' 
training, in addition to current measures of clinical com- 
petence (such as mini-clinical evaluation exercise [18]) 
used in formative and summative assessments of medical 
students? Thirdly, are medical schools failing to assess 
all the appropriate outcomes from their courses by con- 
tinuing to rely too heavily on formal examinations based 
mainly on recall of acquired knowledge and some rea- 
soning? The latter approach has produced many doctors 
who are intellectually and academically prepared for 
their careers, but personal failings, such as poor commu- 
nication skills, lack of empathy and concern for patients, 
motivation, and mental health issues, tend to impinge 
on their work effectiveness. Such failings are typically 



detected too late, when brought to light by examination 
failure or the need for disciplinary action. If it is 
accepted that doctors require more than just academic 
knowledge and technical skills then it makes sense to 
look for additional qualities at the outset and select 
those applicants who already have these qualities or 
seem capable of developing them. 

Conclusions 

The findings of this study as a whole reveal a pattern of 
relationships between cognitive and non-cognitive fac- 
tors and medical school performance consistent with in- 
tuitive and theoretical expectations. Our study suggests 
that measurement of non-cognitive personal qualities in 
applicants could improve the selection of medical stu- 
dents, especially in regard to performance in the inter- 
personal skills and professional behaviours needed by 
doctors. However, further research is needed into the 
best non-cognitive measures for the prediction of vari- 
ous target skills and behaviours, and into the degree to 
which such measures can improve on the predictive val- 
idity of existing cognitive selection measures. If such re- 
search confirms which personal qualities are most 
important, then it may indeed be possible to succeed in 
the aim of producing doctors that meet better the expec- 
tations of twenty-first century patients. 
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